Apply regression target transforms to the unnormalized target [RES-2639] - #1199
Open
bejaeger wants to merge 1 commit into
Open
Conversation
Swaps the two steps of a member's target pipeline: the preset now reshapes the target in its original units, and the result is standardized afterwards. Applied to a z-normalised target, `1_plus_log` was `log1p` of a z-score, which is undefined below -1: for a symmetric target roughly 15% of the rows came out non-finite and reached the model, since the log presets carry no scaler of their own. `safepower` was fitted on standardized values, where yeo-johnson cannot reach the log limit that actually removes skew from a heavy right tail. Also resolves the `"none"` preset to no transform at all -- it is the identity, so composing it only costs a pass over the target -- and fixes the docstring of `_transform_labels_one`, which called its input "unprocessed". This changes the predictions of the v2, v2.5 and v3 defaults, whose ensembles use `safepower` for half of their estimators. The v2.6 default (`"none"`) and every estimator without a target transform are unaffected. Validated on TabArena regression: 13 datasets, 222 splits per arm, 8 estimators, 1332 fits, -0.031% rmse / -0.069% mae / -0.062% 1-r2, better on 9 of 13 datasets, with the gains concentrated on heavy-tailed positive targets (healthcare_insurance_expenses -0.23%, diamonds -0.16%, physiochemical_protein -0.13%). That run measured the same semantics through an earlier implementation of this change; the two agree to 1e-7 relative on the five datasets used to cross-check them. The caveat from that report carries over: dropping the power transform entirely costs only +0.069% rmse on those 13 datasets, so the view caps what this can be worth and the effect sits below its noise floor. Follow-up suggested on the ticket is to validate on skewed positive-target data. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
bejaeger
force-pushed
the
benjamin/res-2639-transform-unnormalized-target
branch
from
August 21, 2026 14:42
553c0ec to
e7d29d7
Compare
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e7d29d7. Configure here.
| # The preset reshapes the target in its own units, and only then is | ||
| # the result standardized. Reshaping a standardized target instead | ||
| # means something else entirely: `1_plus_log` was `log1p` of a | ||
| # z-score, undefined wherever that dropped below -1. |
There was a problem hiding this comment.
Comment narrates prior transform order
Low Severity
The comment on the new pipeline order explains what the previous ordering did differently (1_plus_log as log1p of a z-score). Team convention is that comments describe only current behavior, not prior behaviour or ticket history.
Triggered by learned rule: Docstrings describe current behavior only — no call sites or code history
Reviewed by Cursor Bugbot for commit e7d29d7. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes RES-2639. Stacked on #1198 (targets that branch, so this diff is only the behaviour change); review #1198 first.
Change
One swap, in
make_target_transform: the preset reshapes the target in its original units, and the result is standardized afterwards.Applied to a z-normalised target,
1_plus_logwaslog1pof a z-score — undefined below -1, so for a symmetric target ~15% of rows came out non-finite and reached the model, the log presets carrying no scaler of their own.safepowerwas fitted on standardized values, where yeo-johnson cannot reach the log limit that removes skew from a heavy right tail.Also resolves the
"none"preset to no transform (it is the identity, so composing it only costs a pass over the target), and fixes the_transform_labels_onedocstring, which called its input "unprocessed" — the ticket's second item.Impact
Changes the v2 / v2.5 / v3 defaults, whose ensembles use
safepowerfor half of their estimators. Unaffected: the v2.6 default ("none"),n_estimators=1on any version, and every estimator whose target transform isNone.Validation
TabArena regression, 13 datasets, 222 splits per arm, 8 estimators, checkpoint
x49jl1p6, 1332 fits, no failures (full report on the ticket):Errors, so negative is better. Gains concentrate on heavy-tailed positive targets:
healthcare_insurance_expenses−0.23%,diamonds−0.16%,physiochemical_protein−0.13%.That run measured these semantics through an earlier implementation (
80bf5fa0, which composed the frame into each pipeline instead of moving the standardization). This stack reproduces it to 1e-7 relative on the five datasets used to cross-check, so the validation transfers.Caveat from that report, carried over: dropping the power transform entirely costs only +0.069% rmse on those 13 datasets, so that view caps what any change to how the transform composes can be worth, and this effect sits below its noise floor — for scale, incidental numerical drift between v8.3.0 and base moved one dataset by 0.60%. The direction is right and the mechanism is a bug fix; the magnitude is not certifiable on TabArena regression. Suggested follow-up on the ticket: validate on skewed positive-target data.
A variant that additionally scale-normalises the yeo-johnson transforms was measured in the same run and is not part of this stack — it gave back about half the gain (+0.017% rmse). Parked on
benjamin/res-2639-safepower-scale-normalized.Tests
The test that #1198 pinned to the old ordering is flipped here, which is the reviewable core of the diff. Added:
log1pof the target is finite wherelog1pof a z-score is not, the"none"preset resolves to no transform, and end-to-end accuracy on a skewed target for three presets.🤖 Generated with Claude Code